3 Building Containers

Chapter 2

Author

Ryan Wesslen

From the docs:

Containers are like light-weight virtual machines — container engines use operating system tricks to isolate programs from each other (“containing” them), making them work as though they were running on their own hardware with their own filesystem. This makes execution environments more reproducible, for example by preventing accidental cross-contamination of environments on the same machine. For added security, Modal runs containers using the sandboxed gVisor container runtime.

Containers are started up from a stored “snapshot” of their filesystem state called an image. Producing the image for a container is called building the image.

By default, Modal functions are executed in a Debian Linux container with a basic Python installation of the same minor version v3.x as your local Python interpreter.

Customizing this environment is critical. To make your apps and functions useful, you will probably need some third party system packages or Python libraries. To make them start up faster, you can bake data like model weights into the container image, taking advantage of Modal’s optimized filesystem for serving containers.

Modal provides a number of options to customize your container images at different levels of abstraction and granularity, from high-level convenience methods like pip_install through wrappers of core container image build features like RUN and ENV to full on “bring-your-own-Dockerfile”. We’ll cover each of these in this guide, along with tips and tricks for building images effectively when using each tool.

3.1 import_sklearn.py

3.1.1 Install scikit-learn in a custom image

This builds a custom image which installs the sklearn (scikit-learn) Python package in it. It’s an example of how you can use packages, even if you don’t have them installed locally.

First, the imports:

import time
import modal

Next, we’ll define an app, with a custom image that installs sklearn.

app = modal.App(
    "import-sklearn",
    image=modal.Image.debian_slim()
    .apt_install("libgomp1")
    .pip_install("scikit-learn"),
)

Chaining

A nice design in modal is the idea of method chaining, where the image is built by layers.

The app.image.imports() lets us conditionally import in the global scope. This is needed because we might not have sklearn and numpy installed locally, but we know they are installed inside the custom image.

with app.image.imports():
    import numpy as np
    from sklearn import datasets, linear_model

Now, let’s define a function that uses one of scikit-learn’s built-in datasets and fits a very simple model (linear regression) to it.

@app.function()
def fit():
    print("Inside run!")
    t0 = time.time()
    diabetes_X, diabetes_y = datasets.load_diabetes(return_X_y=True)
    diabetes_X = diabetes_X[:, np.newaxis, 2]
    regr = linear_model.LinearRegression()
    regr.fit(diabetes_X, diabetes_y)
    return time.time() - t0

Finally, we’d trigger the run locally. We also time this. Note that the first time we run this, it will build the image. This might take 1-2 min. When we run this subsequent times, the image is already build, and it will run much much faster.

if __name__ == "__main__":
    t0 = time.time()
    with app.run():
        t = fit.remote()
        print("Function time spent:", t)
    print("Full time spent:", time.time() - t0)

Let’s now run it all:

$ modal run import_sklearn.py 
✓ Initialized. View run at https://modal.com/charlotte-llm/main/apps/ap-xxxxxxxxxx
Building image im-m9EoOtS0dmWsGUat8WCWFc

=> Step 0: FROM base

=> Step 1: RUN apt-get update
Get:1 http://deb.debian.org/debian bullseye InRelease [116 kB]
Get:2 http://deb.debian.org/debian-security bullseye-security InRelease [48.4 kB]
Get:3 http://deb.debian.org/debian bullseye-updates InRelease [44.1 kB]
Get:4 http://deb.debian.org/debian bullseye/main amd64 Packages [8068 kB]
Get:5 http://deb.debian.org/debian-security bullseye-security/main amd64 Packages [275 kB]
Get:6 http://deb.debian.org/debian bullseye-updates/main amd64 Packages.diff/Index [26.3 kB]
Get:7 http://deb.debian.org/debian bullseye-updates/main amd64 Packages T-2023-12-29-1403.39-F-2023-07-31-2005.11.pdiff [6053 B]
Get:7 http://deb.debian.org/debian bullseye-updates/main amd64 Packages T-2023-12-29-1403.39-F-2023-07-31-2005.11.pdiff [6053 B]
Get:8 http://deb.debian.org/debian bullseye-updates/main amd64 Packages [18.8 kB]
Fetched 8602 kB in 4s (2239 kB/s)
Reading package lists...

=> Step 2: RUN apt-get install -y libgomp1
Reading package lists...
Building dependency tree...
Reading state information...
libgomp1 is already the newest version (10.2.1-6).
libgomp1 set to manually installed.
0 upgraded, 0 newly installed, 0 to remove and 46 not upgraded.
Creating image snapshot...
Finished snapshot; took 1.14s

Built image im-m9EoOtS0dmWsGUat8WCWFc in 8.53s
Building image im-Kndkz3TpRhPEMy6UcNR7YR

=> Step 0: FROM base

=> Step 1: RUN python -m pip install scikit-learn 
Looking in indexes: http://pypi-mirror.modal.local:5555/simple
Collecting scikit-learn
  Downloading http://pypi-mirror.modal.local:5555/simple/scikit-learn/scikit_learn-1.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (13.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.3/13.3 MB 169.9 MB/s eta 0:00:00
Requirement already satisfied: numpy>=1.19.5 in /usr/local/lib/python3.10/site-packages (from scikit-learn) (1.25.0)
Collecting scipy>=1.6.0 (from scikit-learn)
  Downloading http://pypi-mirror.modal.local:5555/simple/scipy/scipy-1.13.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (38.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 38.6/38.6 MB 233.1 MB/s eta 0:00:00
Collecting joblib>=1.2.0 (from scikit-learn)
  Downloading http://pypi-mirror.modal.local:5555/simple/joblib/joblib-1.4.2-py3-none-any.whl (301 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 301.8/301.8 kB 252.9 MB/s eta 0:00:00
Collecting threadpoolctl>=3.1.0 (from scikit-learn)
  Downloading http://pypi-mirror.modal.local:5555/simple/threadpoolctl/threadpoolctl-3.5.0-py3-none-any.whl (18 kB)
Installing collected packages: threadpoolctl, scipy, joblib, scikit-learn
Successfully installed joblib-1.4.2 scikit-learn-1.5.0 scipy-1.13.1 threadpoolctl-3.5.0

[notice] A new release of pip is available: 23.1.2 -> 24.0
[notice] To update, run: pip install --upgrade pip
Creating image snapshot...
Finished snapshot; took 2.27s

Built image im-Kndkz3TpRhPEMy6UcNR7YR in 13.14s
✓ Created objects.
├── 🔨 Created mount /modal-examples/02_building_containers/import_sklearn.py
└── 🔨 Created function fit.
Inside run!
Stopping app - local entrypoint completed.
✓ App completed. View run at https://modal.com/charlotte-llm/main/apps/ap-xxxxxxxxxx

So from the above, it took 8.53s to build the first image, 2.27s to create the snapshot and 13.14s to build the second image. But if we run this again, it’ll be much faster than before as we’ve already.

3.2 import_sklearn_r2.py

Just for fun, let’s modify this script to now output the R^2 value on the test data.

import_sklearn_r2.py

import time
import modal

app = modal.App(
    "import-sklearn",
    image=modal.Image.debian_slim()
    .apt_install("libgomp1")
    .pip_install("scikit-learn"),
)

with app.image.imports():
    import numpy as np
    from sklearn import datasets, linear_model
    from sklearn.model_selection import train_test_split
    from sklearn.metrics import r2_score

@app.function()
def fit():
    print("Inside run!")
    X, y = datasets.load_diabetes(return_X_y=True)
    X = X[:, np.newaxis, 2]
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)
    regr = linear_model.LinearRegression()
    regr.fit(X_train, y_train)
    predict = regr.predict(X_test)
    
    return r2_score(predict, y_test)


if __name__ == "__main__":
    t0 = time.time()
    with app.run():
        t = fit.remote()
        print("R Squared is:", t)
    print("Full time spent:", time.time() - t0)

Running this, we get:

$ modal run import_sklearn_r2.py
✓ Initialized. View run at https://modal.com/charlotte-llm/main/apps/ap-xxxxxxxxx
✓ Created objects.
├── 🔨 Created mount /modal-examples/02_building_containers/import_sklearn_r2.py
└── 🔨 Created function fit.
Inside run!
Stopping app - local entrypoint completed.
✓ App completed. View run at https://modal.com/charlotte-llm/main/apps/ap-xxxxxxxxxx

This result somewhat surprised me.

First, I didn’t see the output R^2. I was expecting this perhaps the first time running, but didn’t see it.

Second, after running, unlike the previous example that shut down immediately, this container was running ephemerally:

So let’s rerun, but this time renaming our function from fit to fit_r2:

$ modal run import_sklearn_r2.py
✓ Initialized. View run at https://modal.com/charlotte-llm/main/apps/ap-xxxxxxxxxx
✓ Created objects.
├── 🔨 Created mount /modal-examples/02_building_containers/import_sklearn_r2.py
└── 🔨 Created function fit_r2.
Inside run!
Stopping app - local entrypoint completed.
✓ App completed. View run at https://modal.com/charlotte-llm/main/apps/ap-xxxxxxxxxx

This avoided the issue of perpetually running but didn’t print the R^2 to console:

The main takeaway is to keep an eye on the naming of remote functions.

3.3 install_cuda.py

The next examples shows how to use the Nvidia CUDA base image from DockerHub.

Here’s a list of the different CUDA images available.

We need to add Python 3 and pip with the add_python option because the image doesn’t have these by default.

install_cuda.py

from modal import App, Image

image = Image.from_registry(
    "nvidia/cuda:12.2.0-devel-ubuntu22.04", add_python="3.11"
)
app = App(image=image)

@app.function(gpu="T4")
def f():
    import subprocess

    subprocess.run(["nvidia-smi"])

Running it provides:

$ modal run install_cuda.py     
✓ Initialized. View run at https://modal.com/charlotte-llm/main/apps/ap-xxxxxxxxxx
Building image im-NAV0762Ag7PgTCJY8XyAqb

=> Step 0: FROM nvidia/cuda:12.2.0-devel-ubuntu22.04
Getting image source signatures
Copying blob sha256:9a9dd462fc4c5ca1dd29994385be60a5bb359843fc93447331b8c97dfec99bf9
Copying blob sha256:9fe5ccccae45d6811769206667e494085cb511666be47b8e659087c249083c3f
Copying blob sha256:aece8493d3972efa43bfd4ee3cdba659c0f787f8f59c82fb3e48c87cbb22a12e
Copying blob sha256:bdddd5cb92f6b4613055bcbcd3226df9821c7facd5af9a998ba12dae080ef134
Copying blob sha256:8054e9d6e8d6718cc3461aa4172ad048564cdf9f552c8f9820bd127859aa007c
Copying blob sha256:5324914b447286e0e6512290373af079a25f94499a379e642774245376e60885
Copying blob sha256:95eef45e00fabd2bce97586bfe26be456b0e4b3ef3d88d07a8b334ee05cc603c
Copying blob sha256:e2554c2d377e1176c0b8687b17aa7cbe2c48746857acc11686281a4adee35a0a
Copying blob sha256:4640d022dbb8eb47da53ccc2de59f8f5e780ea046289ba3cffdf0a5bd8d19810
Copying blob sha256:aa750c79a4cc745750c40a37cad738f9bcea14abb96b0c5a811a9b53f185b9c9
Copying blob sha256:9e2de25be969afa4e73937f8283a1100f4d964fc0876c2f2184fda25ad23eeda
Copying config sha256:fead46ae620f9febc59f92a8f1f277f502ef6dca8111ce459c154d236ee84eee
Writing manifest to image destination
Unpacking OCI image
   • unpacking rootfs ...
   • ... done
   • unpacked image rootfs: /tmp/.tmpDUhHRA

=> Step 1: COPY /python/. /usr/local

=> Step 2: RUN ln -s /usr/local/bin/python3 /usr/local/bin/python

=> Step 3: ENV TERMINFO_DIRS=/etc/terminfo:/lib/terminfo:/usr/share/terminfo:/usr/lib/terminfo

=> Step 4: COPY /modal_requirements.txt /modal_requirements.txt

=> Step 5: RUN python -m pip install --upgrade pip
Looking in indexes: http://pypi-mirror.modal.local:5555/simple
Requirement already satisfied: pip in /usr/local/lib/python3.11/site-packages (23.2.1)
Collecting pip
  Obtaining dependency information for pip from http://pypi-mirror.modal.local:5555/simple/pip/pip-24.0-py3-none-any.whl.metadata
  Downloading http://pypi-mirror.modal.local:5555/simple/pip/pip-24.0-py3-none-any.whl.metadata (3.6 kB)
Downloading http://pypi-mirror.modal.local:5555/simple/pip/pip-24.0-py3-none-any.whl (2.1 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 216.4 MB/s eta 0:00:00
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 23.2.1
    Uninstalling pip-23.2.1:
      Successfully uninstalled pip-23.2.1
Successfully installed pip-24.0

=> Step 6: RUN python -m pip install -r /modal_requirements.txt
Looking in indexes: http://pypi-mirror.modal.local:5555/simple
Ignoring cloudpickle: markers 'python_version < "3.11"' don't match your environment
Ignoring ddtrace: markers 'python_version < "3.11"' don't match your environment
Collecting aiohttp==3.8.3 (from -r /modal_requirements.txt (line 2))

...

Creating image snapshot...
Finished snapshot; took 6.10s

Built image im-NAV0762Ag7PgTCJY8XyAqb in 136.43s
✓ Created objects.
├── 🔨 Created mount /modal-examples/02_building_containers/install_cuda.py
└── 🔨 Created function f.

==========
== CUDA ==
==========

CUDA Version 12.2.0

Container image Copyright (c) 2016-2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license

A copy of this license is made available in this container at /NGC-DL-CONTAINER-LICENSE for your convenience.

Thu Jun 13 23:08:03 2024       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.15              Driver Version: 550.54.15      CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  Tesla T4                       On  |   00000000:36:00.0 Off |                 ERR! |
| N/A   32C ERR!               9W /   70W |       0MiB /  15360MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
                                                                                         
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+
Stopping app - local entrypoint completed.
✓ App completed. View run at https://modal.com/charlotte-llm/main/apps/ap-xxxxxxxxxx

While this process did take a few minutes, it’s very easy and should be very quick if rerunning.

It’s also helpful to note how to run a subprocess as we would in Python anyways:

import subprocess

subprocess.run(["nvidia-smi"])

3.4 screenshot.py

Last, we’ll finish

--- title: "Building Containers" subtitle: "Chapter 2" author: "Ryan Wesslen" toc: true format: html: html-math-method: katex code-tools: true self-contained: true execute: echo: true eval: false --- From the [docs](https://modal.com/docs/guide/custom-container): > Containers are like light-weight virtual machines — container engines use operating system tricks to isolate programs from each other (“containing” them), making them work as though they were running on their own hardware with their own filesystem. This makes execution environments more reproducible, for example by preventing accidental cross-contamination of environments on the same machine. For added security, Modal runs containers using the sandboxed gVisor container runtime. > > Containers are started up from a stored “snapshot” of their filesystem state called an image. Producing the image for a container is called building the image. > > By default, Modal functions are executed in a Debian Linux container with a basic Python installation of the same minor version v3.x as your local Python interpreter. > > Customizing this environment is critical. To make your apps and functions useful, you will probably need some third party system packages or Python libraries. To make them start up faster, you can bake data like model weights into the container image, taking advantage of Modal’s optimized filesystem for serving containers. > > Modal provides a number of options to customize your container images at different levels of abstraction and granularity, from high-level convenience methods like pip_install through wrappers of core container image build features like RUN and ENV to full on “bring-your-own-Dockerfile”. We’ll cover each of these in this guide, along with tips and tricks for building images effectively when using each tool. ## import_sklearn.py ### Install scikit-learn in a custom image This builds a custom image which installs the sklearn (scikit-learn) Python package in it. It's an example of how you can use packages, even if you don't have them installed locally. First, the imports: ```python import time import modal ``` Next, we'll define an app, with a custom image that installs `sklearn`. ```python app = modal.App( "import-sklearn", image=modal.Image.debian_slim() .apt_install("libgomp1") .pip_install("scikit-learn"), ) ``` ::: {.callout-tip} ## Chaining A nice design in `modal` is the idea of method chaining, where the image is built by [layers](https://docs.docker.com/build/guide/layers/). ::: The `app.image.imports()` lets us conditionally import in the global scope. This is needed because we might not have sklearn and numpy installed locally, but we know they are installed inside the custom image. ```python with app.image.imports(): import numpy as np from sklearn import datasets, linear_model ``` Now, let's define a function that uses one of scikit-learn's built-in datasets and fits a very simple model (linear regression) to it. ```python @app.function() def fit(): print("Inside run!") t0 = time.time() diabetes_X, diabetes_y = datasets.load_diabetes(return_X_y=True) diabetes_X = diabetes_X[:, np.newaxis, 2] regr = linear_model.LinearRegression() regr.fit(diabetes_X, diabetes_y) return time.time() - t0 ``` Finally, we'd trigger the run locally. We also time this. Note that the first time we run this, it will build the image. This might take 1-2 min. When we run this subsequent times, the image is already build, and it will run much much faster. ```python if __name__ == "__main__": t0 = time.time() with app.run(): t = fit.remote() print("Function time spent:", t) print("Full time spent:", time.time() - t0) ``` Let's now run it all: ```bash $ modal run import_sklearn.py ✓ Initialized. View run at https://modal.com/charlotte-llm/main/apps/ap-xxxxxxxxxx Building image im-m9EoOtS0dmWsGUat8WCWFc => Step 0: FROM base => Step 1: RUN apt-get update Get:1 http://deb.debian.org/debian bullseye InRelease [116 kB] Get:2 http://deb.debian.org/debian-security bullseye-security InRelease [48.4 kB] Get:3 http://deb.debian.org/debian bullseye-updates InRelease [44.1 kB] Get:4 http://deb.debian.org/debian bullseye/main amd64 Packages [8068 kB] Get:5 http://deb.debian.org/debian-security bullseye-security/main amd64 Packages [275 kB] Get:6 http://deb.debian.org/debian bullseye-updates/main amd64 Packages.diff/Index [26.3 kB] Get:7 http://deb.debian.org/debian bullseye-updates/main amd64 Packages T-2023-12-29-1403.39-F-2023-07-31-2005.11.pdiff [6053 B] Get:7 http://deb.debian.org/debian bullseye-updates/main amd64 Packages T-2023-12-29-1403.39-F-2023-07-31-2005.11.pdiff [6053 B] Get:8 http://deb.debian.org/debian bullseye-updates/main amd64 Packages [18.8 kB] Fetched 8602 kB in 4s (2239 kB/s) Reading package lists... => Step 2: RUN apt-get install -y libgomp1 Reading package lists... Building dependency tree... Reading state information... libgomp1 is already the newest version (10.2.1-6). libgomp1 set to manually installed. 0 upgraded, 0 newly installed, 0 to remove and 46 not upgraded. Creating image snapshot... Finished snapshot; took 1.14s Built image im-m9EoOtS0dmWsGUat8WCWFc in 8.53s Building image im-Kndkz3TpRhPEMy6UcNR7YR => Step 0: FROM base => Step 1: RUN python -m pip install scikit-learn Looking in indexes: http://pypi-mirror.modal.local:5555/simple Collecting scikit-learn Downloading http://pypi-mirror.modal.local:5555/simple/scikit-learn/scikit_learn-1.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (13.3 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.3/13.3 MB 169.9 MB/s eta 0:00:00 Requirement already satisfied: numpy>=1.19.5 in /usr/local/lib/python3.10/site-packages (from scikit-learn) (1.25.0) Collecting scipy>=1.6.0 (from scikit-learn) Downloading http://pypi-mirror.modal.local:5555/simple/scipy/scipy-1.13.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (38.6 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 38.6/38.6 MB 233.1 MB/s eta 0:00:00 Collecting joblib>=1.2.0 (from scikit-learn) Downloading http://pypi-mirror.modal.local:5555/simple/joblib/joblib-1.4.2-py3-none-any.whl (301 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 301.8/301.8 kB 252.9 MB/s eta 0:00:00 Collecting threadpoolctl>=3.1.0 (from scikit-learn) Downloading http://pypi-mirror.modal.local:5555/simple/threadpoolctl/threadpoolctl-3.5.0-py3-none-any.whl (18 kB) Installing collected packages: threadpoolctl, scipy, joblib, scikit-learn Successfully installed joblib-1.4.2 scikit-learn-1.5.0 scipy-1.13.1 threadpoolctl-3.5.0 [notice] A new release of pip is available: 23.1.2 -> 24.0 [notice] To update, run: pip install --upgrade pip Creating image snapshot... Finished snapshot; took 2.27s Built image im-Kndkz3TpRhPEMy6UcNR7YR in 13.14s ✓ Created objects. ├── 🔨 Created mount /modal-examples/02_building_containers/import_sklearn.py └── 🔨 Created function fit. Inside run! Stopping app - local entrypoint completed. ✓ App completed. View run at https://modal.com/charlotte-llm/main/apps/ap-xxxxxxxxxx ``` So from the above, it took `8.53s` to build the first image, `2.27s` to create the snapshot and `13.14s` to build the second image. But if we run this again, it'll be much faster than before as we've already. ## import_sklearn_r2.py Just for fun, let's modify this script to now output the R^2 value on the test data. ```{.python filename="import_sklearn_r2.py"} import time import modal app = modal.App( "import-sklearn", image=modal.Image.debian_slim() .apt_install("libgomp1") .pip_install("scikit-learn"), ) with app.image.imports(): import numpy as np from sklearn import datasets, linear_model from sklearn.model_selection import train_test_split from sklearn.metrics import r2_score @app.function() def fit(): print("Inside run!") X, y = datasets.load_diabetes(return_X_y=True) X = X[:, np.newaxis, 2] X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42) regr = linear_model.LinearRegression() regr.fit(X_train, y_train) predict = regr.predict(X_test) return r2_score(predict, y_test) if __name__ == "__main__": t0 = time.time() with app.run(): t = fit.remote() print("R Squared is:", t) print("Full time spent:", time.time() - t0) ``` Running this, we get: ```bash $ modal run import_sklearn_r2.py ✓ Initialized. View run at https://modal.com/charlotte-llm/main/apps/ap-xxxxxxxxx ✓ Created objects. ├── 🔨 Created mount /modal-examples/02_building_containers/import_sklearn_r2.py └── 🔨 Created function fit. Inside run! Stopping app - local entrypoint completed. ✓ App completed. View run at https://modal.com/charlotte-llm/main/apps/ap-xxxxxxxxxx ``` This result somewhat surprised me. First, I didn't see the output R^2. I was expecting this perhaps the first time running, but didn't see it. Second, after running, unlike the previous example that shut down immediately, this container was running ephemerally: ![](images/02_building_containers/sklearn-r2.png) So let's rerun, but this time renaming our function from `fit` to `fit_r2`: ```bash $ modal run import_sklearn_r2.py ✓ Initialized. View run at https://modal.com/charlotte-llm/main/apps/ap-xxxxxxxxxx ✓ Created objects. ├── 🔨 Created mount /modal-examples/02_building_containers/import_sklearn_r2.py └── 🔨 Created function fit_r2. Inside run! Stopping app - local entrypoint completed. ✓ App completed. View run at https://modal.com/charlotte-llm/main/apps/ap-xxxxxxxxxx ``` This avoided the issue of perpetually running but didn't print the R^2 to console: ![](images/02_building_containers/sklearn-r2-2.png) The main takeaway is to keep an eye on the naming of remote functions. ## install_cuda.py The next examples shows how to use the Nvidia CUDA base image from DockerHub. Here's a [list](https://hub.docker.com/r/nvidia/cuda/tags) of the different CUDA images available. We need to add Python 3 and pip with the `add_python` option because the image doesn't have these by default. ```{.python filename="install_cuda.py"} from modal import App, Image image = Image.from_registry( "nvidia/cuda:12.2.0-devel-ubuntu22.04", add_python="3.11" ) app = App(image=image) @app.function(gpu="T4") def f(): import subprocess subprocess.run(["nvidia-smi"]) ``` Running it provides: ```bash $ modal run install_cuda.py ✓ Initialized. View run at https://modal.com/charlotte-llm/main/apps/ap-xxxxxxxxxx Building image im-NAV0762Ag7PgTCJY8XyAqb => Step 0: FROM nvidia/cuda:12.2.0-devel-ubuntu22.04 Getting image source signatures Copying blob sha256:9a9dd462fc4c5ca1dd29994385be60a5bb359843fc93447331b8c97dfec99bf9 Copying blob sha256:9fe5ccccae45d6811769206667e494085cb511666be47b8e659087c249083c3f Copying blob sha256:aece8493d3972efa43bfd4ee3cdba659c0f787f8f59c82fb3e48c87cbb22a12e Copying blob sha256:bdddd5cb92f6b4613055bcbcd3226df9821c7facd5af9a998ba12dae080ef134 Copying blob sha256:8054e9d6e8d6718cc3461aa4172ad048564cdf9f552c8f9820bd127859aa007c Copying blob sha256:5324914b447286e0e6512290373af079a25f94499a379e642774245376e60885 Copying blob sha256:95eef45e00fabd2bce97586bfe26be456b0e4b3ef3d88d07a8b334ee05cc603c Copying blob sha256:e2554c2d377e1176c0b8687b17aa7cbe2c48746857acc11686281a4adee35a0a Copying blob sha256:4640d022dbb8eb47da53ccc2de59f8f5e780ea046289ba3cffdf0a5bd8d19810 Copying blob sha256:aa750c79a4cc745750c40a37cad738f9bcea14abb96b0c5a811a9b53f185b9c9 Copying blob sha256:9e2de25be969afa4e73937f8283a1100f4d964fc0876c2f2184fda25ad23eeda Copying config sha256:fead46ae620f9febc59f92a8f1f277f502ef6dca8111ce459c154d236ee84eee Writing manifest to image destination Unpacking OCI image • unpacking rootfs ... • ... done • unpacked image rootfs: /tmp/.tmpDUhHRA => Step 1: COPY /python/. /usr/local => Step 2: RUN ln -s /usr/local/bin/python3 /usr/local/bin/python => Step 3: ENV TERMINFO_DIRS=/etc/terminfo:/lib/terminfo:/usr/share/terminfo:/usr/lib/terminfo => Step 4: COPY /modal_requirements.txt /modal_requirements.txt => Step 5: RUN python -m pip install --upgrade pip Looking in indexes: http://pypi-mirror.modal.local:5555/simple Requirement already satisfied: pip in /usr/local/lib/python3.11/site-packages (23.2.1) Collecting pip Obtaining dependency information for pip from http://pypi-mirror.modal.local:5555/simple/pip/pip-24.0-py3-none-any.whl.metadata Downloading http://pypi-mirror.modal.local:5555/simple/pip/pip-24.0-py3-none-any.whl.metadata (3.6 kB) Downloading http://pypi-mirror.modal.local:5555/simple/pip/pip-24.0-py3-none-any.whl (2.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 216.4 MB/s eta 0:00:00 Installing collected packages: pip Attempting uninstall: pip Found existing installation: pip 23.2.1 Uninstalling pip-23.2.1: Successfully uninstalled pip-23.2.1 Successfully installed pip-24.0 => Step 6: RUN python -m pip install -r /modal_requirements.txt Looking in indexes: http://pypi-mirror.modal.local:5555/simple Ignoring cloudpickle: markers 'python_version < "3.11"' don't match your environment Ignoring ddtrace: markers 'python_version < "3.11"' don't match your environment Collecting aiohttp==3.8.3 (from -r /modal_requirements.txt (line 2)) ... Creating image snapshot... Finished snapshot; took 6.10s Built image im-NAV0762Ag7PgTCJY8XyAqb in 136.43s ✓ Created objects. ├── 🔨 Created mount /modal-examples/02_building_containers/install_cuda.py └── 🔨 Created function f. ========== == CUDA == ========== CUDA Version 12.2.0 Container image Copyright (c) 2016-2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. This container image and its contents are governed by the NVIDIA Deep Learning Container License. By pulling and using the container, you accept the terms and conditions of this license: https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license A copy of this license is made available in this container at /NGC-DL-CONTAINER-LICENSE for your convenience. Thu Jun 13 23:08:03 2024 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | |-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 Tesla T4 On | 00000000:36:00.0 Off | ERR! | | N/A 32C ERR! 9W / 70W | 0MiB / 15360MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ +-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | No running processes found | +-----------------------------------------------------------------------------------------+ Stopping app - local entrypoint completed. ✓ App completed. View run at https://modal.com/charlotte-llm/main/apps/ap-xxxxxxxxxx ``` While this process did take a few minutes, it's very easy and should be very quick if rerunning. It's also helpful to note how to run a subprocess as we would in Python anyways: ```python import subprocess subprocess.run(["nvidia-smi"]) ``` ## screenshot.py Last, we'll finish